Managing web page data in a composite document

ABSTRACT

A composite document system is implemented as instructions stored on a computer-readable medium. When executed on a processor, the instructions cause the processor to receive model data that describes a three-dimensional model of an object, where the three-dimensional model is generated using a modeling software, and where the model data conforms to a non-image format of the modeling software. The instructions further cause the processor to receive web page data that includes content and description of the content in a mark-up language, generate a composite document that includes the model data and the web page data, interpret the model data in the composite document to generate a visual representation of the three-dimensional model, and interpret the web page data in the composite document to generate a visual representation of the web page data.

FIELD OF TECHNOLOGY

This disclosure generally relates to generating and managing a compositedocument and, in particular, to generating and managing a compositedocument that includes web page data.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent it is described in thisbackground section, as well as aspects of the description that may nototherwise qualify as prior art at the time of filing, are neitherexpressly nor impliedly admitted as prior art against the presentdisclosure.

Computer-generated presentations are widely used in business, academic,scientific, and other environments. Typically, a computer-generatedpresentation is made up of multiple slides having text and, in somecases, images in a digital image file format such as BMP, JPEG, etc. Forexample, a user may illustrate the presentation by inserting drawings orphotographs into the corresponding slides. Generally, during apresentation, the user sequentially shows the slides to an audience.

SUMMARY

In an embodiment, a tangible non-transitory computer-readable mediumstores instructions thereon to implement a composite document system.When executed on a processor, the instructions cause the processor toreceive model data that describes a three-dimensional model of anobject, where the three-dimensional model is generated using a modelingsoftware, and where the model data conforms to a non-image format of themodeling software. The instructions further cause the processor toreceive web page data that includes content and description of thecontent in a mark-up language, generate a composite document thatincludes the model data and the web page data, interpret the model datain the composite document to generate a visual representation of thethree-dimensional model, and interpret the web page data in thecomposite document to generate a visual representation of the web pagedata.

In another embodiment, a tangible non-transitory computer-readablemedium stores instructions thereon that, when executed on a processor,cause the processor to provide a set of modeling controls via a userinterface, where a user utilizes the set of modeling controls to draw athree-dimensional model of an object and generate model data descriptiveof the three-dimensional model in accordance with commands received fromthe user interface. The instructions further cause the processor toprovide a web reference control via the user interface, where the userspecifies a source web page using the web reference control, receive webpage data, where the web page data is retrieved from the source web pageand includes content and description of the content, and generate acomposite document using the model data and the web page data.

In another embodiment, a method in a computing device includes receivingmodel data that conforms to a non-image format, where the model data isstored on a computer-readable medium, and where the model data describesa three-dimensional model, receiving web page data that includes contentand instructions, where the web page data is retrieved from a source webpage, generating a composite document using the model data and the webpage data, and causing the composite document to be displayed on a userinterface. Causing the composite document to be displayed includesgenerating a visual representation of the three-dimensional model usingthe model data and generating a visual representation of the source webpage using the web page data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computing environment in which a systemfor generating and managing composite documents (“the composite documentsystem”) operates, according to an embodiment;

FIG. 2 a block diagram of an example composite document system that mayoperate in the computing environment of FIG. 1;

FIG. 3 is a block diagram of an example web application engine that maybe used in the composite document system of FIG. 2;

FIG. 4 is an example screenshot of a user interface window which the webapplication engine of FIG. 3 may generate to receive a reference to webpage data;

FIG. 5 is an example screenshot of a user interface window which the webapplication engine of FIG. 3 may generate to receive positioning datafor web page data;

FIG. 6 is an example screenshot of a user interface window which the webapplication engine of FIG. 3 may generate to receive a clipping mask forweb page data;

FIG. 7 is an example screenshot of a user interface window which the webapplication engine of FIG. 3 may generate to allow a user to inspect webpage data included in a composite document;

FIG. 8 is an example screenshot of a user interface window which the webapplication engine of FIG. 3 may generate to allow a user to edit acomposite document that includes modeling data and web page data;

FIG. 9 is a flow diagram of an example method for generating a compositedocument that includes web page data;

FIG. 10 is a flow diagram of an example method for automaticallyretrieving updated web page data to be included in a composite document;

FIG. 11 is a block diagram of an example satellite and aerial dataengine that may be used in the composite document system of FIG. 2;

FIG. 12 is an example screenshot of a user interface window which thesatellite and aerial data engine of FIG. 11 may generate to receive areference to satellite and aerial image data;

FIG. 13 is an example screenshot of a user interface window which thesatellite and aerial data engine of FIG. 11 may generate to receivepositioning data for satellite and aerial image data;

FIG. 14 is an example screenshot of a user interface window which thesatellite and aerial data engine of FIG. 11 may generate to configure acomposite document that includes satellite and aerial image data;

FIG. 15 is a flow diagram of an example method for generating acomposite document that includes modeling data and satellite and aerialimage data;

FIG. 16 is a block diagram of an example map data engine that may beused in the composite document system of FIG. 2;

FIG. 17 is an example screenshot of a user interface window which themap data engine of FIG. 16 may generate to receive a reference to mapdata;

FIG. 18 is an example screenshot of a user interface window which themap data engine of FIG. 16 may generate to receive positioning data formap data;

FIG. 19 is an example screenshot of a user interface window which themap data engine of FIG. 16 may generate to allow a user to edit acomposite document that includes map data;

FIG. 20 is a flow diagram of an example method for generating acomposite document that includes modeling data and map data; and

FIG. 21 is a flow diagram of an example method for processing potentialreferences to non-text based entities for use in a composite document.

DETAILED DESCRIPTION

Generally speaking, a system for creating and managing compositedocuments (“the composite document system”) allows a user to efficientlygenerate a composite document that includes several types of data, suchas text, images, video and/or audio content, web page data, map data,satellite and aerial data. The composite document system further allowsthe user to edit a previously created composite document, update thecomposite document with new data of one or more types, and adjust visualcharacteristics of the data included in the composite document. In anembodiment, the composite document system is integrated with a modelingsoftware for creating three-dimensional (3D) models of objects such asbuildings, vehicles, furniture, etc., so that a 3D model developed usingthe modeling software can be added to a composite document as model datathat conforms to a non-image format of the modeling software. A user canmanipulate (e.g., rotate, scale, spin about a certain point or “orbit”)the 3D model in the composite document system without launching themodeling software as a separate application or generating an instance ofthe modeling software within a window embedded in the composite documentsystem, for example. Further, in an embodiment, the composite documentsystem provides a set of controls on a user interface for drawingtwo-dimensional (2D) shapes. A user may use a composite document systemthat implements some or all of the techniques of the present disclosureto create an interactive presentation made up of a series ofpresentation slides, for example.

In at least some of the embodiments discussed herein, compositedocuments include multiple data objects (or “entities”) that arerendered in corresponding containers. An entity may include anindication of the type of data (e.g., text, an image, a 2D drawingshape, a 3D model, web page data, map data, satellite and aerial imagedata), the geometry (e.g., positioning, orientation) of the visualrepresentation of the data in the composite document, and a reference tothe source of the data (such as a URL referring to the web page fromwhich web page data was received, for example). Thus, for example, acertain composite document may contain a model entity that includes a 3Dmodel of a building and positioning data that indicates where acontainer for the model is located when the composite document isdisplayed on a user interface, as well as a text entity that includes atext describing the model and the corresponding positioning data.

According to some embodiments, a user can interact with and modify datawithin containers by invoking functionality resident in the compositedocument system. For example, in an embodiment, the composite documentsystem receives and stores web page data, including content and adescription of the content in a mark-up language, from a host disposedon the Internet. The user then spatially locates a container for the webpage data within a composite document, and the composite document systeminterprets the web page data and generates an appropriate visualrepresentation (or simply “image”) of the web page data. Further, insome embodiments, the visual representation of the web page data isinteractive, and the user may activate links included in the web pagedata, click on buttons, launch playback of audio and video content, andotherwise interact with the web page data. To this end, the compositedocument system may include a set of application programming interface(API) functions of an embedded browser, such as Cocoa NSWebView used ona Mac OS platform or the WebBrowser class used on a Windows platform,for example.

Further, for at least some types of data, the composite document systemprovides a set of interactive tools that allow a user to specify sourcesof data to be used in various entities (e.g., URLs of web pages,references to geographic locations for which map data or satellite andaerial imagery is obtained, references to 3D models), inspect entities,edit entities, activate functions within certain entities (e.g.,activate a Flash animation in web page data), update entities withup-to-date information (e.g., retrieve updated web page data, a newtraffic map, or an updated 3D model), reposition or re-orient containersthat enclose entity data, etc. Some of these interactive tools mayoperate on only some of the types of entities, while other tools mayoperate on entities of all types. Further, some of the tools may be madeavailable to a user in accordance with the current mode of operation(e.g., edit, presentation).

Several example systems and methods for generating and managingcomposite documents that include one or more of web page data, map data,and satellite image data are discussed below. First, a brief overview ofan example composite document system and an environment in which thecomposite document system may operate is discussed with reference toFIGS. 1 and 2. Several techniques for including web page data in acomposite document are then discussed with reference to FIGS. 3-10,followed by a discussion of several techniques for including satelliteand aerial image data with reference to FIGS. 11-15, further followed bya discussion of several techniques for including map data in a compositedocument with reference to FIGS. 16-19.

System Overview

FIG. 1 is a block diagram of a computing environment 10 in which anexample composite document system 100 may operate to allow users todynamically select and include data of different types in compositedocuments. In the embodiment of FIG. 1, the composite document creationsystem 100 operates in a computing device 102 and includes a webapplication engine 104, a 3D modeling application engine 106, asatellite and aerial data engine 105, a map data engine 107, and a textapplication engine 108 stored as computer-readable instructions on atangible, non-transitory storage medium (or “program storage”) 110.However, in other embodiments, the composite document creation system100 includes only some of the components 104-108. For example, in oneembodiment, the composite document creation system 100 includes the 3Dmodeling application engine 106 and the web application engine 104, butdoes not include the satellite and aerial data engine 105 or the mapdata engine 107. In another embodiment, the composite document system100 includes the 3D modeling application engine 106 and the satelliteand aerial data engine 105, but does not include the web applicationengine 104 or the map data engine 107. In yet another embodiment, thecomposite document creation system 100 includes the 3D modelingapplication engine 106 and the map data engine 107, but does not includethe web application engine 104 or the satellite and aerial data engine105. In general, the composite document creation system 10 may includeany suitable combination of the components 104-108.

In operation, the composite document creation system 100 receives inputfrom an input sub-system 112 communicatively coupled to the computingdevice 102. The input sub-system 112 generally may include one or moreof a pointing device such as a mouse, a keyboard, a touch screen, atrackball device, a digitizing tablet, a microphone, etc. In theembodiment of FIG. 1, the input sub-system 112 includes a mouse 116 anda keyboard 114. Further, the composite document creation system 100provides output, such as a visual representation of a compositedocument, prompts and dialogs, etc., to an output sub-system 120 that isalso communicatively coupled to the computing device 102. In anembodiment, the output sub-system 120 includes an LCD display monitor124 and a plotter 122, although in other embodiments, the outputsub-system 120 additionally or alternatively includes one or more of atouch screen, printer, a speaker, etc. In some embodiment, the computingdevice 102 includes one or both of the input sub-system 112 and theoutput system 120. As discussed in more detail below, the compositedocument creation system 100 provides a graphical user interface (GUI)to a user via the input sub-system 112 and the output sub-system 120.

With continued reference to FIG. 1, the computing device 102 may includea central processing unit (CPU) 126, a program memory 128, and a datastorage 129. Depending on the embodiment, the program memory 128 may bea Dual Data Rate Dynamic Random Access Memory (DDR-DRAM), as illustratedin FIG. 1, a Static Random Access Memory (SRAM), or any other suitabletype of a non-persistent high-speed memory. The program memory 128 maybe provided on the same chip as the CPU 126 or on a separate chip. Thedata storage 129 may be a hard disk drive (HDD) that communicates withthe CPU 126 over a bus, for example, or another type of a suitablepersistent storage device. In operation, the CPU 126 retrieves thecomputer instructions that make up the engines 104-108 from the datastorage 129 and executes the instructions to provide the functionalityof the engines 104-108.

The composite document system 100 may include one or several networkcards (not shown) and communicate with remote hosts disposed on anetwork 132 (e.g., the Internet) via a communication link 130 (e.g., anEthernet link). For example, the web application engine 104 may receiveweb page data from a source web page provided by a web server 134, themap data engine 107 may receive map data from a map data server 142, andthe satellite and aerial data engine 105 may receive satellite andaerial data from a satellite and aerial data server 144. In anotherembodiment, the map data server 142 and the satellite and aerial dataserver 144 operate on a same host. Further, in an embodiment, some orall of the engines 104-108 can be downloaded to the computing device 102from an application download server 146.

In the embodiment of FIG. 1, the computing device 102 is a personalcomputer (PC). However, in other embodiments, the computing device 102may be any suitable stationary or portable computing device such as atablet PC, a smartphone, etc. Further, although the computing device 102in the example embodiment of FIG. 1 includes both storage and processingcomponents that can fully support the composite document system 100, thecomputing device 102 in other embodiments can be a so-called thin clientthat depends on another computing device for certain computing and/orstorage functions. For example, in one such embodiment, the data storage129 and the program storage 110 are external to the computing device 102and are connected to the computing device 102 via the Ethernet link 130.

Now referring to FIG. 2, the composite document system 100 of FIG. 1 maybe implemented, for example, as a composite document system 200 thatexecutes in a software environment 201. In an embodiment, a webapplication engine 202, a satellite and aerial data engine 203, a 3Dmodeling engine 204, a map data engine 205, a text engine 206, and a 2Ddrawing engine 207 are software modules that interface with anapplication framework or wrapper 208 of the document system 200. Theengines 202-206 may be similar to the respective ones of the engines104-108 of FIG. 1. Further, depending on the embodiment, the compositedocument system 200 may include additional components such as an imagerendering engine (not shown) to render data that conforms to a standardimage format such as BMP, JPEG, GIF, etc. Conversely, in someembodiments, the composite document system 200 may include only some ofthe engines 202-207. The application wrapper 208 may operate as a “glue”layer and provide APIs, variables, message queues, etc. using which theengines 202-207 and possibly other components of the composite documentsystem 200 may interact with each other. The application wrapper 208 mayin turn interface with an operating system (OS) 210 such as Android,Linux, Windows, Mac OS, etc. In another embodiment, the compositedocument system 200 executes on a Java Virtual Machine (JVM) in a Javaenvironment, and the application wrapper 208 includes functions that areplatform-agnostic, i.e., independent of the operating system and thehardware on which the operating system executes. In yet anotherembodiment, some or all of the components of the composite documentsystem 200 are implemented in a scripting language interpreted by abrowser, for example.

In operation, the OS 210 allows the engines 202-207 to receive inputdata from input devices and provide output to output devices, such asdevices operating in the input sub-system 112 and the output sub-system120 of FIG. 1, respectively. In particular, the OS 210 may include oneor several user interface APIs 222 to access input and output devices,input device drivers 224, such as a mouse driver 228 and keyboard driver230, as well as output device drivers 226, such as a graphic videodriver 232 and a printer driver 234, for example. Further, some or allof the engines 202-206 may programmatically interface with a networkstack 218. Referring back to FIG. 1, in an embodiment, the network stack218 enables tasks executing in the software environment 201 tocommunicate with remote hosts such as servers 134, 136, 142, 144 and146. For example, the network stack 218 may implement the TCP/IPprotocol and interface with a network driver 220 that controls a networkcard, via which tasks in the software environment 201 can send andreceive data.

The text engine 206 may process text input received from an input deviceor another document, for example. In an embodiment, the text engine 206supports styled text editing (e.g., Rich Text Format or “RTF”) thatallows users to apply to the text various fonts, font faces (e.g., bold,italic, light, heavy), styles, paragraph formatting (e.g., left-align,right-align), tabs, tabular layout, bulleted and/or number lists, etc.

In an embodiment, a 3D modeling software 240 for developing 3D models ofvarious objects operates in the software environment 201. The 3Dmodeling software 240 generally provides a set of modeling controls togenerate, position, and variously adjust three-dimensional shapes, applytextures to the shapes or surfaces, define interactions between shapes,etc. Models developed using the 3D modeling software 240 may be storedon a computer-readable medium as data files that conform to a certainnon-image format. For example, the non-image format may specify a set offaces of a 3D models along with the corresponding attributes, such asthe position and orientation of a face, the texture of the face, etc.According to an embodiment, the 3D modeling engine 204 is configured torender a 3D model developed using the 3D modeling software 240 andstored in the non-image format of the 3D modeling software 240. Further,in some embodiments, the 3D modeling engine 204 is configured to enableadditional interaction with the 3D model within a correspondingcontainer included in a composite document. The 3D modeling engine 204may allow a user to rotate, scale, and otherwise select and adjust themanner in which the 3D model is rendered, for example. Still further, insome embodiments, the 3D modeling engine 204 may provide at leastpartial editing functionality for a 3D model included in the compositedocument. In one such embodiment, the 3D modeling engine 204 is a sharedsoftware component, such as a Dynamic-link Library (DLL) that includes aset of functions which both the composite document system 200 and the 3Dmodeling software 240 access during operation.

With continued reference to FIG. 2, the composite document system 200may also interface with DLLs or other function libraries that provideadditional functionality to one or more software applications operatingin the software environment 201. For example, in an embodiment, a webview library 242 provides embedded browser functionality. The web viewlibrary 242 may include, for example, functions for generating andprocessing data packets that conform to the Hypertext Transfer Protocol(HTTP), encode data according to the Secure Socket Layer (SSL) protocol,request and verify digital certificates, etc.

Further, in some embodiments, the composite document system 200 mayinteract with one or more plugin components (not shown) that variouslyextend the functionality of an embedded browser operating in thecomposite document system 200. For example, a Flash plugin may beinstalled in the software environment 201 as a DLL to support playbackof Flash-based content, a plugin for interacting with street-levelphotographs and videos provided by a corresponding service may beinstalled in the software environment 201 as another DLL, etc.

As discussed in more detail below, the web application engine 202receives web page data from a remote host such as the web server 134illustrated in FIG. 1. Web page data may include content, such as text,images, animations, video files, audio files, etc. as well asinstructions in a mark-up language such as HTML. The web applicationengine 202 may also automatically retrieve content referenced in the webpage data using URLs, for example. In an embodiment, the web applicationengine 202 is configured to interpret the content of web page dataaccording to the mark-up language and generate a visual representationof the interpreted content in a corresponding container. In someembodiments, the web application engine 202 may include or invoke thefunctionality of the web view library 242 to provide an embedded browserwithin the corresponding container. A user may accordingly view the webdata and interact with the web data in at least a limited manner (e.g.,activate Flash content included in the web page data, scroll through thecontent of the web page data, activate “forward” and “back” controls toretrieve the corresponding data, when available).

The map data engine 205 provides user interface controls for a user tospecify a geographic location and, upon receiving location data from theuser interface, receives map and related cartographic data from a remotehost such as the map data server 142 illustrated in FIG. 1, according toan embodiment. The map data engine 205 may invoke appropriate APIsprovided by the wrapper 208 to interface with the network stack 212and/or APIs provided by a map data service. For example, an API maycause a request for map data corresponding to a particular type (e.g.,terrain) for a particular location to be transmitted to a map dataserver and, when the map data is received, render the map data withinthe corresponding container.

Further, in an embodiment, the satellite and aerial data engine 203receives a selection of a geographic location from a user interface andretrieves satellite and aerial data corresponding to the selectedgeographic location. Similar to the map data engine 205, the satelliteand aerial data engine 203 may invoke APIs provided by a satellite andaerial data service. In an embodiment, the satellite and aerial dataservice is provided by a satellite and aerial image data server such asthe server 144 of FIG. 1, for example.

In an embodiment, some or all of the engines 202-207 communicate with amass storage driver 236 to store and retrieve composite documents from apersistent memory such as the data storage 129 of FIG. 1, for example.In an embodiment, the web application engine 202 stores web page data aswell as objects embedded in the web page data and/or referenced by theweb page data. In general, a composite document generated by thecomposite document system 200 may be stored in any suitable format suchas extensible markup language (XML), for example. According to someembodiments or scenarios, a composite document is stored as several datafiles (e.g., an XML file, one or several audio file, one or several 3Dmodel files, etc.).

A composite document may be stored with a static snapshot of dynamicdata included in the composite document, according to some embodiments.For example, a composite document may include the content of a certainweb page as well as the images referenced in the content. As anotherexample, a composite document may store both a 3D model in a non-imageformat of the 3D modeling software 240 and a snapshot of the 3D model ina standard digital image format.

Generally speaking, a user may utilize the composite document system 100or 200 to create an interactive presentation made up of a series ofpresentation slides, edit a previously created presentation, and displaythe presentation slides to an audience. The interactive presentation mayinclude data of only one type or, in some cases, no data at all. In someembodiments, the interactive presentation may include two or more of a3D model, 2D shapes, text, images in a digital image format, web pagedata, map data, satellite and aerial image data, etc., each of which maybe stored in a respective entity and rendered in a respective container.In some embodiments, the composite document system 100 or 200 operatesin several modes, such as an edit mode and a presentation mode, and indifferent modes provides different functionality and different sets ofuser controls to a user. For example, a user may be able to rotate a 3Dmodel included in a composite document both in the edit mode and in thepresentation mode, but resizing the container that includes the 3D modelmay be available only in the edit mode.

Next, several features that may be provided by the web applicationengine 202, the satellite and aerial data engine 203, and the map dataengine 205 are discussed in the respective sections below.

Web Application Engine

FIG. 3 is a block diagram of an example web application engine 300 thatmay be implemented as a set of computer-executable instructions andoperate in a composite document system, such as the composite documentsystem 100 or 200. Referring back to FIG. 2, the web application engine300 may operate in the composite document system 200 as the webapplication engine 202 or a component of the web application engine 202,for example. The web application engine 300 may be configured to receivea reference to a source web page or a file containing web page data,receive web page data from the source web page or file, generate a webreference entity, store the web reference entity and/or the web pagedata in a computer-readable memory, provide users controls for editingthe web page data, update the web page data, generate a visualrepresentation of the web page data, enclose and render the web pagedata in a corresponding container included in a composite document, etc.In the embodiment of FIG. 3, the web application engine 300 includes aweb reference tool 302, a web reference editor 304, a web referenceinspector tool 306, and a web reference renderer 308. A user may be ableto access some of the tools 302, 304, and 306 only in certain operationmodes, according to an embodiment. In another embodiment, the tools 302,304, and 306 may provide only partial functionality to a user in certainmodes of operation.

As illustrated in FIG. 3, the web reference tool 302 may include a webreference selector tool 314 and a web reference geometry tool 316.During operation, the web reference selector tool 314 may receive a URLof a source web page (or another reference to web page data) from a userinterface and receive web page data in accordance with the URL. The webreference geometry tool 316 may locate the received web page data withina container in a composite document and receive an initial geometry(e.g., positioning, orientation, size) of the container. The containermay appear, for example, as a rectangle in which an embedded browser isdisplayed. In some scenarios, the URL of the source web page indicates aresource at a remote web server, while in other scenarios, the URLindicates a file stored in a local file system. In the edit mode, theweb reference tool 302 may receive a URL from the user interface, followthe links embedded in the web page corresponding to the URL (whenpossible), and render web page content, possibly including dynamic mediacontent such as animation, audio, video, etc.

The web application engine 300 may generate a web reference entity 320as a data structure (or “object”) that includes the web page data aswell as the URL of the source web page from which the web page data wasretrieved. The web reference entity 320 may also include data objectsreferenced by the web page data, such as images, videos, tabular data,etc. In an embodiment, the web reference entity 320 also includes astatic snapshot of the web page data, e.g., the content of a web page,the description of the content in a mark-up language, and the imagesreferenced in the content. Additionally or alternatively, the staticsnapshot of the web page includes an image of the web page data in adigital image format, for example. The web reference entity 320 may bestored in a memory such as the data storage 129 (see FIG. 1), forexample.

The web reference editor 304 may provide an interface for an embeddedweb browser, so that a user can interact with the web page data receivedby the web reference tool 302 via a set of controls similar to thoseprovided by a typical browser application (e.g., “back” and “forward”browser buttons and a “refresh” button). In an embodiment, the webreference editor 304 also allows the user to edit the URL of the webpage data as well as the content in the web page data. Referring back toFIG. 2, the web reference editor 304 may interact with the web viewlibrary 242 or similar functions. When activated in the presentationmode, the web reference editor 304 may allow the user to manipulate theweb page data and view dynamic media content such as animation, audio,video, etc. An example interface which the web reference editor 304 mayprovide, and the related functionality of the web reference editor 304,are discussed in more detail with reference to FIG. 8.

In an embodiment, the web application engine 300 automaticallydetermines whether a new version of web page data is available at a hostfrom which the web page was retrieved, according to an embodiment. Theweb application engine 300 accordingly may notify the user via anappropriate dialog box or automatically retrieve the new version of theweb page data, depending on the embodiment or configuration.

With continued reference to FIG. 3, the web reference renderer 308includes a vector graphics engine 310 and a raster graphics engine 312,according to an embodiment. When the web reference tool 302 receives acommand to adjust one or more characteristics of the visualrepresentation of the web page data, such as a command to zoom in orout, rotate the container that includes the visual representation of theweb page data, or move the container to a new location, the vectorgraphics engine 310 may re-render the web page data in accordance withthe received command to generate an updated visual representation of theweb page data. Further, the web reference tool 302 may appropriatelyscale the updated visual representation to the size of the web page datacontainer in the composite document. The raster graphics engine 312 mayconvert the output of the vector graphics engine 310 to a bit-map imagesized to the container of the web page data and display the bit mapimage in the composite document.

FIG. 4 is an example screenshot of a user interface window 400 which theweb reference selector tool 314 of FIG. 3 generates so that a user mayenter a reference to web page data. In an embodiment, the web referenceselector tool 314 causes a dialog box 402 to be displayed at an outputdevice in response to receiving an indication that the Insert command404 has been selected in the File menu 406. According to an embodiment,the web reference selector tool 314 is available in the edit mode when acomposite document is being developed, and is not available in thepresentation mode. The composite document in the example of FIG. 4 alsoincludes a 3D model entity that contains model data, rendered in theuser interface window 400 as a 3D model 409.

The web reference selector tool 314 may receive the URL of a remote webpage (or another resource identifier) via an input text box 408 in anysuitable manner (from a keyboard, from the OS clipboard, etc.). Inresponse to detecting that an open button 410 has been activated, theweb reference selector tool 314 may parse the URL received via the textbox 408 and initiate the retrieval of web page data from thecorresponding web page. The dialog box 402 may also include a browsebutton 412, so that a user may invoke a local file browser and a selectweb page data from a local resource, e.g., a file located in the filesystem of a computing device in which the web application engine 300operates. Further, in an embodiment, the selector tool 314 verifies thereceived data to determine whether the received data conforms to arecognized format of web page data, such as HTML.

In an embodiment, the web reference selector tool 314 displays thedialog box 402 or a similar dialog box upon determining that certaintext in an OS clipboard includes a URL. The web application engine 300may prompt the user to determine whether the identified text should beinterpreted as a URL and a web reference entity should be created withweb page data corresponding to the URL. For example, when a user wishesto transfer text from a text editing software to a document compositesystem in which the web reference selector tool 314 operates, the textediting software may place the text into the OS clipboard, and the webreference selector tool 314, in response to the user pasting the textfrom the OS clipboard into a composite document, may automaticallygenerate a text entity to contain the text. However, when the textincludes a string of characters that parse into a URL, the compositedocument system may suggest to the user that a web reference entity begenerated in addition to, or instead of, the text entity. The compositedocument system may further suggest to the user that web page data beretrieved from the URL detected in the pasted text.

FIG. 5 is an example screenshot of a user interface window 500 which theweb reference geometry tool 316 of FIG. 3 may generate to receivepositioning and geometry data from a user interface and position acontainer 504 within a composite document 502. The container 504encloses an image that corresponds to a visual representation of webpage data. In the example embodiment of FIG. 5, the web referencegeometry tool 316 implements a rotate tool 506, a move tool 508, and atilt tool 510 to allow the user to rotate the container 504 or the imageenclosed within the container 504, move the container 504 to a newlocation within the composite document 50, and tilt the container 504 orthe image enclosed within the container 504, respectively. The webreference geometry tool 316 may modify the size and shape of thecontainer 504 in response to a user actuating proportional dimensioncontrols 512 or horizontal/vertical dimension controls 514 using apointing device, for example.

Referring to FIG. 6, the web reference editor 304 of FIG. 3 or a similarcomponent may generate a user interface window 600 to allow a user toclip an image corresponding to received web page data. For example, auser may utilize a shape tool 602, provided as a component of thegeometry tool 316 and accessible via a menu bar 604, to define a desiredshape, which may be regular or irregular, for the image or a certainsection of the image. In the example of FIG. 6, a user defines acircular clipping mask 606 for the image corresponding to web page data608 using commands provided in a pull-down menu 612. In an embodiment,web application engine 300 stores the definition of the clipping mask606 as a part of the web reference entity corresponding to the web pagedata 608.

FIG. 7 is an example screenshot of a user interface window 700 which theweb application engine 300 may generate when a new version of web pagedata is available at a source (e.g., a host on the Internet) from whichthe currently available version of the web page has been retrieved,according to an embodiment. The web application engine 300 may activatethe web reference inspector tool 306 when a user selects the WebReference Inspector 704 menu option from a menu bar 702 or automaticallyupon entering the edit mode, for example. In an embodiment, the webreference inspector tool 316 receives user input from an inspectordialog 706 that may include a text box 708, a refresh button 712, a savebutton 716, and an undo button 718. As indicated above, a web referenceentity may include the URL of the source web page data along with thecorresponding web page data, and the inspector tool 316 mayautomatically display the URL of the source web page as pre-filled textin the text box 708. The user may update the URL as necessary, requestthat the web application engine 300 retrieve a new version of the webpage data by clicking on the refresh button 712 and request that the webapplication engine 300 save new version of the web page data in the webreference entity by clicking on the save button 716. The inspectordialog 706 may be dockable and modeless, i.e., non-blocking with respectto other windows, dialogs, containers, and tools. Further, in someembodiments, the inspector dialog 706 also includes “back,” “forward,”and other browser buttons or controls.

As further illustrated in FIG. 7, the user interface window 700 mayrender the currently available web page data in a container 720. In anembodiment, the web application engine 300 receives a new version of webpage data, compares the newly received web page data with the currentversion of the web page data in the corresponding web reference entityand, if the newly received web page data is different from the currentversion of the web page data, the web application engine 300 generates anotification icon 724.

FIG. 8 is an example screenshot of a user interface window 800 which theweb reference editor 304 of FIG. 3 may generate to allow users to editweb page data. The web reference editor 304 may provide an html editor802 as an embedded window in which the web page data is displayed. Asillustrated in FIG. 8, the displayed web page data includes content(e.g., text) as well instructions in a mark-up language such as HTML. Insome scenarios, web page data may include images, Flash content, etc.,and the html editor 802 may display appropriate icons to indicate thepresence of such content. The html editor 802 may also parse themodified web page data and generate an appropriate notification if themodified web page data does not comply with the HTML language.

As further illustrated in FIG. 8, the web reference editor 304 mayinterpret and render the web page data in a container 804. For example,the web reference editor 304 may invoke an embedded browser such as theweb view library 242 illustrated in FIG. 2. The web reference editor 304may additionally provide navigation controls in a toolbar 805 to allow auser to actively interact with rendered web page data. In an embodiment,when the user clicks on an update button 806, the web reference editor304 may generate an image corresponding to the modified web page dataand display image in the container 804. However, if the user clicks onthe undo button 808, the web reference editor 304 discards changes madeto the web page data via the html editor 802. The web reference editor304 may store the changes to the web page data in the corresponding webreference entity when the users clicks on the save button 810.

Next, an example method for generating a composite document thatincludes web page data and at least one other type of data is discussedwith reference to FIG. 9. By way of example only, FIG. 9 illustrates amethod 900 for generating a composite document that includes web pagedata and 3D modeling data. However, it is noted that methods similar tothe method 900 may be implemented for generating composite documentshaving other types of data in addition to web page data, e.g., 2Ddrawing data, text data, image data, map data, etc.

At block 902, 3D model data is received in, for example, a non-imageformat in which the corresponding 3D model was developed using anothersoftware application (such as the 3D modeling software 240 illustratedin FIG. 2). A model entity may be generated that includes, for example,the received model data, an indication of the type of received data, areference to the source of the 3D model data, etc. Next, at block 904,web page data is received. For example, in an embodiment, an address ofa source web page is received from a user interface, and web page datais automatically retrieved from the source web page. A web referenceentity may be generated that includes, for example, the received webpage data and the address of the source web page. At block 906, acomposite document that includes the model data and the web page datareceived at blocks 902 and 904, respectively, is generated. Thecomposite document accordingly may include a model entity and a webreference entity. In other scenarios, a composite document also mayinclude text entities, image entities, sound entities, etc.

At block 908, the entities included in the composite document arerendered. To this end, in an embodiment, each of the modeling data andthe web page data is interpreted according to the format to which thecorresponding data conforms. More specifically, in an embodiment, themodeling data conforms to a non-image format of a 3D modeling software,and the modeling data is interpreted at block 908 using an appropriate3D modeling engine, so as to generate a raster image or a vector-basedimage. Further, the web page data may be interpreted and rendered atblock 908 according to the instructions in the mark-up language such asHTML.

According to an embodiment, the method 900 is implemented in thecomposite document system 200. For example, the method 900 may be atleast partially implemented in the 3D modeling engine 204 and the webapplication engine 202. Further, to implement the method 900, the webreference tool 302 may be used to facilitate user interaction withrespect to web page data.

FIG. 10 is a flow diagram an example method 950 for automaticallyretrieving updated web page data for an existing web reference entity.At block 952, a request to update web page is received. Depending on theembodiment, the request is received from a user interface (e.g., from auser interface screen presented by reference inspector tool 306) orautomatically in response to a timer expiration, for example. Therequest may specify the URL that identifies a source web page from whichthe current version of web page was received. In some scenarios, the URLis stored in the originally generated web reference entity. In otherscenarios, the user may specify a new URL (using the web referenceselector tool 314, for example).

Next, at block 954, web page data is retrieved in accordance with therequest received at block 952. At block 956, the newly retrieved webpage data is compared to the web page data already available in the webreference entity to determine whether the web page has changed. In somescenarios, web page data includes an indicator of when the web page waslast updated (e.g., day, hour, minute), and the indicator in the newlyreceived web page data is compared to a similar indicator in thepreviously stored web page data. In another embodiment, the content inthe new web page data is compared to the content in the previouslystored web page data.

If it is determined that the web page data has been updated, the newversion of the web page data is stored in the corresponding webreference entity at block 958. Otherwise, if it is determined that thesame web page data has been retrieved at block 954, the flow of themethod 950 bypasses block 958, and the method 950 completes.

Satellite and Aerial Image Data Engine

As indicated above, a composite document system may also allow users togenerate, edit, and present composite documents that include interactivesatellite and aerial image data. According to some embodiments, asatellite data service provides one or both of satellite and aerialimage data via a data server (such as the satellite and aerial dataserver 144 illustrated in FIG. 1) as well as APIs for accessing,rendering, and interacting with such data. Satellite and aerial imagedata may be provided in a manner that allows rendering of a geographicarea in 3D and simulating movement through 3D space. For example, asatellite image of a mountain range may be projected onto, or otherwisecombined with, a 3D terrain view of the same area. Further, a satellitedata service may provide terrain data as well as geo-relevant staticdata such as street names and dynamic data such as current weather data,traffic data, and airline flight tracking data. The additionalgeo-relevant data may be associated with one or several layers to enableselective display of such data over satellite and aerial image data. Forsimplicity, data that includes one or both of satellite imagery andaerial imagery, as well as additional geo-relevant data, is referred tobelow as “satellite data.” In some embodiments, satellite data mayconform to the Keyhole Markup Language (KML) format or, when distributedin a compressed manner, to the KML-zipped (KMZ) format. In otherembodiments, satellite data conforms to another non-image format. In anembodiment, the composite document system utilizes Google Earth APIservice available at http://code.google.com/apis/earth.

Referring to FIG. 11, an example satellite and aerial data engine 1100may operate in the composite document system 100 or 200, for example, tosupport including, editing, and interacting with satellite data in acomposite document. The satellite and aerial image engine may 1100 mayinclude a satellite image reference tool 1102, a satellite imagereference editor (or tool) 1104, a satellite image reference inspectortool 1106, and a satellite image reference renderer 1108. Further, inthe embodiment illustrated in FIG. 11, the reference renderer 1108includes a raster graphics engine 1110 and a vector graphics engine1112, and the satellite image reference tool 1102 includes a satelliteimage reference selector 1114 and a satellite image reference geometrytool 1116. A user may be able to access some of the tools 1102, 1104,and 1106 only in certain modes of operation, according to an embodiment.

The satellite and aerial data engine 1100 may operate on a satelliteimage reference entity 1120 stored on a computer-readable medium. Thesatellite image reference entity 1120 may include satellite data andsuch additional information as, for example, an indication of ageographic location to which the satellite data corresponds (e.g.,latitude/longitude coordinates, a postal code, a street address), atimestamp to indicate when the satellite data was last updated, asnapshot of the satellite data in a digital image format (e.g., JPEG,PNG, GIF), etc. Similar to the web reference entity 320 discussed above,the satellite image reference entity 1120 may be included in a compositedocument that also includes 3D modeling data or 2D drawing data, storedin a corresponding model or drawing entity, and rendered in acorresponding container when the composite document is displayed on auser interface. Further, the composite document may also include text,images in a digital image format, web page data, map data, etc.

In operation, the satellite image reference selector 1114 may receive areference to a geographic location for which a user wishes to receivesatellite data. In an embodiment, the reference includes geographiccoordinates such as Global Positioning Service (GPS) coordinates orUniversal Transverse Mercator (UTM) coordinates. In other embodiments,the reference includes a street address, a postal code such as the ZoneImprovement Plan (ZIP) code used in the United States, or any othersuitable indication of a geographic location or region. The satelliteimage reference selector 1114 may receive a reference to a geographiclocation from an input device, a file, or the OS clipboard, for example.Upon receiving a reference to a geographic location via the satelliteimage reference selector 1114, the satellite and aerial data engine 1100may request corresponding satellite data using an API provided by asatellite data service, for example. In response to receiving therequested satellite data, the satellite and aerial data engine 1100 maygenerate a new entity (such as the satellite image reference entity1120) to store the satellite data.

The satellite image reference geometry tool 1116 may provide a set ofcontrols for user to define and locate within a composite document acontainer that includes a visual representation of satellite data. Forexample, according to an embodiment, the satellite image referencegeometry tool 1116 receives positioning, size, orientation, and otherinitial geometry data for the container. In those embodiments thatinclude 3D rendering, the initial geometry may also include tiltselection. The initial geometry data may be stored in the correspondingsatellite image reference entity.

With continued reference to FIG. 11, the satellite image referencerenderer 1108 may render satellite data as a bit map image. In anembodiment, the vector graphics engine 1112 generates a vector-basedvisual representation of satellite data using APIs provided by thesatellite data service or by otherwise interpreting satellite datareceived from the satellite data service. The raster graphics engine1110 then converts the output of the vector graphics engine 1110 to abit map image sized according to the dimensions of the container inwhich the satellite data is displayed within the composite document,according to an embodiment.

The satellite image reference renderer 1108 may render and re-rendersatellite data in accordance with user commands received from a userinterface or in response to other events (timer expiration, detectedchanges in source satellite data, etc.). For example, when the satelliteimage reference tool 1102 receives a command to adjust visualcharacteristics of satellite data, such as a command to resize thecorresponding container, the satellite image reference renderer 1108 mayaccordingly re-render the satellite data. Further, the satellite imagereference renderer 1108 may automatically re-render satellite data upondetecting that a user has updated the selection of layers ofgeo-relevant data. For example, when current visual representation ofsatellite data includes weather data, and a user unselects the weatherdata rendering using the satellite image inspector tool 1106, thesatellite image reference renderer 1108 re-renders the satellite data soas to remove weather information. Still further, in an embodiment, thesatellite and aerial data engine 1100 automatically requests certaindynamic data, such as current weather data, for example, according to acertain configurable schedule.

The satellite image inspector tool 1106 provides a set of satellite datacontrols in dockable, modeless window, via which a user selects viewingoptions for a selected satellite data entity, according to anembodiment. The satellite data controls may include an input boxspecifying a reference to a geographic location, navigation controls(such as pan, zoom, rotate, tilt, etc.), and layer visibility controls(e.g., respective toggle switches for current weather data, airlineflight tracking, etc.). In an embodiment, the satellite image inspectortool 1106 also provides 3D viewing controls such as a tilt tool, forexample. In some scenarios, a user utilizes the satellite imageinspector tool 1106 when working with the satellite image referenceeditor 1104.

The satellite image reference editor 1104 may implement a satellite dataviewer configured to render satellite data and support user interactionwith the satellite data. For example, the image reference editor 1104may provide a set of controls via a user interface to allow the user toselect a location in the visual representation of the satellite data,zoom in and out of the selected location, pan to a new location, rotatethe visual representation of the satellite data, etc. In someembodiments, the satellite image reference editor 1104 is available inthe edit mode of a composite document but unavailable in thepresentation mode, for example. In another embodiment, all functions ofthe satellite image reference editor 1104 are available in all modes ofoperation. In yet another embodiment, the satellite image referenceeditor 1104 is available in all modes of operation with differentfunctions enabled.

In an embodiment, the satellite image reference editor 1104 includes asatellite data viewer that is provided as a plugin component. The plugincomponent may be configured to interact with one or more applications inthe corresponding software environment. For example, some or all of thefunctionality of the plugin for a satellite data viewer may beaccessible to the satellite and aerial data engine 1100 as well as abrowser application. As another example, referring back to FIG. 2, theplugin for a satellite data viewer may be installed in the computingenvironment 201 and configured to be accessible to the web applicationengine 202 and the satellite and aerial image data engine 203.

FIG. 12 is an example screenshot of a user interface window 1200 whichthe satellite image reference selector 1114 of FIG. 11 may display toreceive a reference to a geographic location from a user. In anembodiment, a reference selector tool dialog box 1206 is generated whena user selects the Insert command 1204 from the File menu 1202. The usermay specify the reference to a geographic location by supplying in thetext box 1208 one or more of the coordinates, the postal code, thestreet address, etc. of a region for which the user wishes to retrievesatellite data. The user may then click on the button 1212 to initiatethe process of retrieving satellite data. The composite document in theexample of FIG. 12 also includes a 3D model entity that contains modeldata, rendered in the user interface window 1200 as a 3D model 1214.

In an embodiment, the satellite image reference selector 1114 displaysthe dialog box 1206 upon determining that certain text in an OSclipboard potentially includes a reference to a geographic location. Thesatellite and aerial data engine 1100 may prompt the user to determinewhether the identified text should be interpreted as a reference to ageographic location and a satellite image reference entity should becreated with satellite data corresponding to the reference. As discussedabove with reference to the web reference selector tool 314 of FIG. 3,when a user wishes to transfer text from a text editing software to adocument composite, the text editing software may place the text intothe OS clipboard, and the composite document system, in response to theuser pasting the text from the OS clipboard into a composite document,may automatically generate a text entity that includes the text.However, when the text includes a string of characters that can beinterpreted as a street address, an intersection of several streets, apostal code, GPS coordinates or other geographic coordinates, etc., thecomposite document system may suggest to the user that a satellite imagereference entity be generated in addition to, or instead of, the textentity. The composite document system may further suggest to the userthat satellite data be retrieved according to the reference to ageographic location detected in the pasted text.

FIG. 13 is an example screenshot of a user interface window 1300 whichthe satellite image reference geometry tool 1116, discussed withreference to FIG. 11, may generate to receive positioning and geometrydata from a user and position a container 1302 enclosing satellite datadata within a composite document 1303, according to an embodiment. Thesatellite image reference geometry tool 1116 may provide a set ofgeometry and positioning controls such as rotate tools 1306 and 1310, amove tool 1308, proportional dimension controls 1312 to set the size ofthe container 1302, horizontal/vertical dimension controls 1314 tostretch the visual representation of satellite data in the container1302, etc. In some embodiments, the geometry and positioning controlsmay also include 3D rendering controls such as a tilt tool (not shown),for example.

Referring to FIG. 14, a user may select the Satellite ReferenceInspector 1404 item from a menu 1402 provided in a user interface window1400 and, in response, the satellite and aerial data engine 1100 maygenerate a dockable, modeless inspector dialog 1406 to provide severalcontrols for interacting with satellite data in a composite document. Inthe example embodiment illustrated in FIG. 14, the inspector dialog 1406includes a text box 1408 to receive a street address (or anotherreference to a geographic location) and a set of checkboxes 1410 toselect additional data to be displayed over a visual representation ofsatellite data. For example, the checkboxes 1410 allow the user toselect whether one or several components of weather information (e.g.,clouds, forecast) are to be displayed over satellite photographs, andoverlay selection options 1412 further allow the user to apply customoverlay selections (e.g., traffic and street labels displayed over asatellite photograph).

In general, the actions a user may perform via the inspector dialog 1406apply to a currently selected satellite image reference entity. In theexample of FIG. 14, satellite data of the currently selected satelliteimage reference entity is displayed in a container 1416. In anembodiment, the inspector dialog 1406 is accessible in the edit mode ofoperation but not in the presentation mode.

Further, at least in the edit mode of operation, a user may alsoactivate the satellite image reference editor 1104 that displays aneditor toolbar 1418. In an embodiment, the user uses the editor toolbar1418 to change the zoom level, re-center, adjust the orientation, adjustthe tilt, and otherwise interact with the satellite data in thecontainer 1416. To this end, the editor toolbar 1418 may include a zoomselector 1422, a pan control 1424, an orientation control 1428, and atilt control 1430, for example.

According to an example scenario, when editing a composite document, auser specifies a geographic location using the satellite image referenceselector 1114, defines and positions a container within a compositedocument in which satellite data corresponding to the geographiclocation is to be displayed using the satellite image reference geometrytool 1116, and selects the desired characteristics for the visualrepresentation of the satellite data and the desires types of data to beincluded in the visual representation using the editor toolbar 1418and/or the inspector dialog 1406. The satellite data and the relatedparameters, such as the geometry of the corresponding container andoverlay selections, may be stored in a satellite image reference entity.As discussed above, the composite document may include any number ofother entities in addition to the satellite image reference entity.During a presentation, the satellite data may be displayed in one orseveral presentation slides in accordance with the parameters andoptions specified in the edit mode. In some embodiments, the user mayinteract with the satellite data in the presentation mode to zoom in onthe location being discussed in the corresponding presentation slide,“fly over” from one location to another, etc. Moreover, depending on theuser configuration, information such as airline flight tracking data maybe updated in real-time during a presentation. On the other hand, if thecomposite document system cannot access the satellite data provider, thesnapshot included in the satellite image reference entity may bedisplayed in the corresponding container (such as the container 1416)during a presentation.

Referring to FIG. 15, a method 1450 for generating a composite documentthat includes satellite data and 3D modeling data may be implemented inthe composite document system 200. For example, the method 1450 may beat least partially implemented in the 3D modeling engine 204 and thesatellite and aerial image data engine 203. It is noted that generatinga composite document that includes satellite data and 3D modeling datais discussed by way of example only, and that in general, methodssimilar to the method 1450 may be implemented to generate compositedocuments having other types of data in addition to satellite data,e.g., 2D drawing data, text data, image data, web page data, 2D mapdata, etc.

At block 1452, 3D model data is received in, for example, a non-imageformat in which the corresponding 3D model was developed using anothersoftware application (such as the 3D modeling software 240 illustratedin FIG. 2), and a model entity is generated that includes the receivedmodel data. Next, at block 1454, satellite data is received. Forexample, in an embodiment, a reference to a geographic location isreceived from a user interface, and satellite data corresponding to thegeographic location is automatically retrieved from a satellite dataserver (e.g., the satellite and aerial image data server 144 of FIG. 1).A satellite image reference entity may be generated that includes thereceived satellite data and the reference to the geographic location. Atblock 1456, a composite document that includes the model data and thesatellite data received at blocks 1452 and 1454, respectively, isgenerated. The composite document accordingly may include a model entityand a satellite image reference entity. In other scenarios, thecomposite document also may include text entities, image entities, soundentities, etc.

At block 1458, the entities included in the composite document arerendered. To this end, in an embodiment, each of the modeling data andthe satellite data is interpreted according to the format to which therespective data conforms. For example, the modeling data that conformsto a non-image format of a 3D modeling software is interpreted at block1458 so as to generate a raster image or a vector-based image, and thesatellite data that conforms to a non-image format of a satellite dataservice may be interpreted and rendered at block 1458 so as to generatea raster image or a vector-based image.

Map Data Engine

In some cases, a user may wish to include map data in a compositedocument system. In an embodiment, a map data service providesinteractive map data via a data server (such as the map data server 142illustrated in FIG. 1) as well as APIs for accessing, rendering, andinteracting with the map data. The map data may include, for example,topographic data, street and road map data, etc. Further, map data mayinclude additional static or dynamic data that optionally can bedisplayed over the topographic or street data. For example, additionalstatic map data may include such data as bicycle trails, points ofinterest such as museums or theaters, business data such as restaurantsor shops, etc. On the other hand, additional dynamic data may includecurrent traffic data, weather data, etc. The additional static anddynamic map data may be associated with one or several layers to enableselective display of such data. In some embodiments, the map dataservice provides map data as a set of tiles in a digital image format.In other embodiments, the map data service provides map data in anon-image format, and the composite document system interprets the mapdata to generate a visual representation of the corresponding map. Inone such embodiment, the map data service provides an appropriate API togenerate an image based on map data in a non-image format. In anembodiment, the composite document system utilizes APIs provided byGoogle Maps at http://code.google.com/apis/maps/index.html or Mapquestat http://developer.mapquest.com/.

Referring to FIG. 16, an example map data engine 1500 may operate in thecomposite document system 100 or 200, for example, to support including,editing, and interacting with map data in a composite document. The mapdata engine may 1500 may include a map reference tool 1502 having a mapreference selector 1514 and a map reference geometry tool 1516, a mapreference editor 1504, a map reference inspector tool 1506, and a mapreference renderer 1508 having a raster graphics engine 1510 and avector graphics engine 1512. In general, the map data engine 1500 mayoperate on a map reference entity 1520 stored on a computer-readablemedium. The map reference entity 1520 may include map data and suchadditional information as, for example, an indication of a geographiclocation to which the map data corresponds (e.g., latitude/longitudecoordinates, a postal code, a street address), a timestamp to indicatewhen the map data was last updated, a snapshot of the map data in adigital image format (e.g., JPEG, PNG, GIF), etc. Similar to the webreference entity 320 and the satellite image reference entity 1120discussed above, the map reference entity 1520 may be included in acomposite document that also includes 3D modeling data or 2D drawingdata, stored in a corresponding model or drawing entity, and rendered ina corresponding container when the composite document is displayed on auser interface. Further, the composite document may also include text,images in a digital image format, web page data, satellite data, etc.

In an embodiment, the map reference selector 1514 operates similar tosatellite image reference selector 1114 and receives a reference to ageographic location for which a user wishes to receive map data. Thereference may include GPS or UTM coordinates, a street address, a ZIPcode, etc. The map data engine 1100 may then request corresponding mapdata using an API provided by a map data service, for example, and, inresponse to receiving the requested map data, generate a new mapreference entity to store the map data.

The map reference geometry tool 1516 may provide a set of controls suchthat a user can supply positioning, size, orientation, and othergeometry data for a container that encloses a visual representation ofthe map data. The initial geometry data may be stored in thecorresponding map reference entity, such as the map reference entity1520. With continued reference to FIG. 16, the map reference renderer1508 may render map data as a bit map image, and generally may besimilar to the satellite image reference renderer 1108 discussed withreference to FIG. 11. In particular, according to at least someembodiments, the vector graphics engine 1512 generates a vector-basedvisual representation of map data using APIs provided by the map dataservice or by otherwise interpreting map data received from the map dataservice. The raster graphics engine 1512 then converts the output of thevector graphics engine 1510 to a bit map image sized according to thedimensions of the container in which the map data is displayed withinthe composite document, according to an embodiment.

The map reference renderer 1508 may render and re-render map data inaccordance with user commands received from a user interface or inresponse to other events (timer expiration, detected changes in sourcesatellite data, etc.). For example, when the map reference tool 1502receives a command to adjust visual characteristics of map data, such asa command to resize the corresponding container, the map referencerenderer 1508 may accordingly re-render the map data. Further, the mapreference renderer 1508 may automatically re-render map data upondetecting that a user has selected new rendering options. For example, auser may select bicycle trail display to cause map reference renderer1508 to re-render the map data with bicycle trails displayed over thecorresponding map. Further, in an embodiment, the map data engine 1500periodically requests dynamic data, such as current traffic information,and the map reference renderer 1508 re-renders the map data using thenew dynamic data.

The map inspector tool 1506 provides a set of map data controls indockable, modeless window, via which a user selects viewing options fora selected map data entity, according to an embodiment. The map datacontrols may include an input box specifying a reference to a geographiclocation, navigation controls (such as pan, zoom, rotate, etc.), layervisibility controls (e.g., respective toggle switches for currentweather data, current traffic data, airline flight tracking, etc.). Insome scenarios, a user utilizes the map inspector tool 1506 when workingwith the map reference editor 1504.

The map reference editor 1504 may implement a satellite data viewerconfigured to render map data and support user interaction with the mapdata. For example, the image reference editor 1404 may provide a set ofcontrols via a user interface to allow the user to select a location inthe visual representation of the map data, zoom in and out of theselected location, pan to a new location, rotate the visualrepresentation of the map data, etc. In some embodiments, the mapreference editor 1504 is available in the edit mode of a compositedocument but unavailable in the presentation mode, for example. Inanother embodiment, all functions of the map reference editor 1504 areavailable in all modes of operation. In yet another embodiment, the mapreference editor 1504 is available in all modes of operation withdifferent functions enabled.

In an embodiment, the satellite image reference editor 1504 includes amap viewer that is provided as a plugin component, which may beconfigured to interact with one or more applications in thecorresponding software environment. For example, some or all of thefunctionality of the plugin for a map viewer may be accessible to themap data engine 1500 as well as a browser application. As anotherexample, referring back to FIG. 2, the plugin for a map viewer may beinstalled in the computing environment 201 and configured to beaccessible to the web application engine 202 and the map data engine205.

Referring to FIG. 17, the map reference selector 1514 generates a userinterface window 1550 similar to the window 1200 illustrated in FIG. 12to receive a reference to a geographic location from a user. Further,similar to the satellite image reference selector 1114, upon determiningthat certain text in an OS clipboard potentially includes a reference toa geographic location, the map reference selector 1514 may prompt theuser to determine whether a map reference entity should be generatedcorresponding to the detected reference. In an embodiment, upondetecting a potential reference to a geographic location, the compositedocument system allows the user to select between processing thedetected potential reference as text, a reference to a map, a referenceto a satellite and aerial image, or a reference to both a map andsatellite and aerial image.

Referring to FIG. 18, the map reference geometry tool 1516 may generatea user interface window 1600 to receive positioning and geometry datafrom a user and position a container 1602 enclosing map data within acomposite document 1603, according to an embodiment. The map referencegeometry tool 1516 may provide a set of geometry and positioningcontrols such as a rotate tool 1606, a move tool 1608, proportionaldimension controls 1612 to set the size of the container 1602,horizontal/vertical dimension controls 1614 to stretch the visualrepresentation of satellite data in the container 1602, etc.

Now referring to FIG. 19, a user may select the Map Reference Inspector1704 item from a menu 1702 provided in a user interface window 1700 and,in response, the map data engine 1500 may generate a dockable, modelessinspector dialog 1706 to provide several controls for interacting withmap data in a composite document. In the example embodiment illustratedin FIG. 19, the inspector dialog 1706 includes a text box 1708 toreceive a street address (or another reference to a geographic location)and a set of checkboxes 1710 to select additional data to be displayedover a visual representation of map data. For example, the checkboxes1710 allow the user to select for display one or more types of staticinformation such as photographs and transit information and dynamicinformation such as traffic.

In general, the actions a user may perform via the inspector dialog 1706apply to a currently selected map entity. In the example of FIG. 19, mapdata of the currently selected map reference entity is displayed in acontainer 1716. In an embodiment, the inspector dialog 1706 isaccessible in the edit mode of operation but not in the presentationmode.

Further, at least in the edit mode of operation, a user may alsoactivate the map reference editor 1504 that displays an editor toolbar1718. In an embodiment, the user uses the editor toolbar 1718 to changethe zoom level, re-center, adjust the orientation, and otherwiseinteract with the map data in the container 1716. To this end, theeditor toolbar 1718 may include a zoom selector 1722, a pan control1724, and an orientation control 1728.

According to an example scenario, when editing a composite document, auser specifies a geographic location using the map reference selector1514, defines and positions a container within a composite document inwhich map data corresponding to the geographic location is to bedisplayed using the map reference geometry tool 1516, and selects thedesired characteristics for the visual representation of the satellitedata and the desires types of data to be included in the visualrepresentation using the editor toolbar 1718 and/or the inspector dialog1706. The map data and the related parameters, such as the geometry ofthe corresponding container and overlay selections, may be stored in amap reference entity. As discussed above, the composite document mayinclude any number of other entities in addition to the map referenceentity. During a presentation, the map data may be displayed in one orseveral presentation slides in accordance with the parameters andoptions specified in the edit mode. In some embodiments, the user mayinteract with the map data in the presentation mode to zoom in on thelocation being discussed in the corresponding presentation slide, panfrom one location to another, etc. Moreover, depending on the userconfiguration, information such as traffic data may be updated inreal-time during a presentation. On the other hand, if the compositedocument system cannot access the map data provider in the presentationmode, the snapshot included in the map image reference entity may bedisplayed in the corresponding container (such as the container 1716).

Referring to FIG. 20, a method 1800 for generating a composite documentthat includes map data and 3D modeling data may be implemented in thecomposite document system 200. For example, the method 1800 may be atleast partially implemented in the 3D modeling engine 204 and the mapdata engine 205. It is noted that generating a composite document thatincludes map data and 3D modeling data is discussed by way of exampleonly, and that in general, methods similar to the method 1800 may beimplemented to generate composite documents having other types of datain addition to map data, e.g., 2D drawing data, text data, image data,web page data, satellite and aerial image data, etc.

At block 1802, 3D model data is received in, for example, a non-imageformat in which the corresponding 3D model was developed using anothersoftware application (such as the 3D modeling software 240 illustratedin FIG. 2), and a model entity is generated that includes the receivedmodel data. Next, at block 1804, map data is received. For example, inan embodiment, a reference to a geographic location is received from auser interface, and map data corresponding to the geographic location isautomatically retrieved from a map data server (e.g., the map dataserver 142 of FIG. 1). A map reference entity may be generated thatincludes, for example, the received map data and the reference to thegeographic location. At block 1806, a composite document that includesthe model data and the map data received at blocks 1802 and 1804,respectively, is generated. The composite document accordingly mayinclude a model entity and a map reference entity. In other scenarios,the composite document also may include text entities, image entities,sound entities, etc.

At block 1808, the entities included in the composite document arerendered. To this end, in an embodiment, each of the modeling data andthe map data is interpreted according to the format to which therespective data conforms. For example, the modeling data that conformsto a non-image format of a 3D modeling software is interpreted at block1808 so as to generate a raster image or a vector-based image, and themap data that conforms to a non-image format of a map data service maybe interpreted and rendered at block 1808 so as to generate a rasterimage or a vector-based image.

Processing Clipboard Data

As discussed above with reference to each of the web application engine300, the satellite and aerial data engine 1100, and the map data engine1500, text in the OS clipboard in some cases includes strings that canbe parsed as URLs, geographic references such as street addresses,coordinates, or zip codes, etc. By way of further example, FIG. 21illustrates a method 1900 for automatically detecting and processingpotential non-text based entities for use in a composite document. In anembodiment, the method 1900 is implemented in the composite documentsystem 200. In general, the method 1900 may be applied to any memorybuffer that includes text, including an OS clipboard.

At block 1902, text is received from the clipboard of an operatingsystem in which the corresponding composite document engine executes.For example, a user may copy a block of text from a text editingapplication, a browser application, a mailer application, a chatapplication, etc., and paste the copied text into a composite document.The text also may be parsed at block 1902. If it is determined at block1904 that the text includes a potential reference to a non-text entity,the flow proceeds to block 1906 if the potential reference is a webreference or to block 1908 if the potential reference is a geographicreference. Otherwise, if the text does not include any potentialreferences to non-text entities, the flow proceeds to block 1910, wherea text entity is generated to store the text pasted into the compositedocument from the OS clipboard.

At block 1906, the user is asked to confirm that the detected potentialreference to a web page should be processed as a web reference. Forexample, the detected potential reference to a web page may be a stringthat begins with “http://” or “https://.” If the user confirms that thestring should be processed as a web reference, a web reference entity iscreated at block 1912. The web reference entity may include the detectedweb reference and web page data provided by a source web page which theweb reference identifies. Thus, according to an embodiment, web pagedata is automatically retrieved when the user confirms that the stringshould be processed as a web reference. On the other hand, if the userdoes not confirm that the string should be processed as a web reference,the flow proceeds to block 1910.

At block 1908, the user is asked to confirm that the detected potentialreference to a geographic location should be processed as a geographicreference. For example, the detected potential reference to a geographiclocation may be a string that potentially includes a street address, apair of coordinates, etc. If the user confirms that the string should beprocessed as a geographic reference, the user may be further prompted toselect between satellite data and map data. If the user selectssatellite data, a satellite image reference entity is generated at block1920. In some embodiments, the user may be allowed to select bothoptions. If the user selects map data, a map reference entity isgenerated at block 1922. In an embodiment, satellite data and/or mapdata are retrieved according to the user selection and included in thecorresponding entity. If the user does not confirm that the stringshould be processed as a geographic reference, the flow proceeds toblock 1910, and the string is processed as the text entity (or a portionof a text entity).

Additional Considerations

The following additional considerations apply to the foregoingdiscussion. Throughout this specification, plural instances mayimplement components, operations, or structures described as a singleinstance. Although individual operations of one or more methods areillustrated and described as separate operations, one or more of theindividual operations may be performed concurrently, and nothingrequires that the operations be performed in the order illustrated.Structures and functionality presented as separate components in exampleconfigurations may be implemented as a combined structure or component.Similarly, structures and functionality presented as a single componentmay be implemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter discussed in this disclosure.

Certain embodiments are described in this disclosure as including logicor a number of components, modules, or mechanisms. Modules mayconstitute either software modules (e.g., code embodied on amachine-readable medium or in a transmission signal) or hardwaremodules. A hardware module is tangible unit capable of performingcertain operations and may be configured or arranged in a certainmanner. In example embodiments, one or more computer systems (e.g., astandalone, client or server computer system) or one or more hardwaremodules of a computer system (e.g., a processor or a group ofprocessors) may be configured by software (e.g., an application orapplication portion) as a hardware module that operates to performcertain operations as described in this disclosure.

Unless specifically stated otherwise, discussions in this disclosurethat use words such as “processing,” “computing,” “calculating,”“determining,” “presenting,” “displaying,” or the like may refer toactions or processes of a machine (e.g., a computer) that manipulates ortransforms data represented as physical (e.g., electronic, magnetic, oroptical) quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used in this disclosure, any reference to “one embodiment” or “anembodiment” means that a particular element, feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment. The appearances of the phrase “in oneembodiment” in various places in the specification are not necessarilyall referring to the same embodiment.

As used in this disclosure, the terms “comprises,” “comprising,”“includes,” “including,” “has,” “having” or any other variation thereof,are intended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of the “a” or “an” are employed to describe elementsand components of the embodiments discussed in this disclosure. This isdone merely for convenience and to give a general sense of theinvention. This description should be read to include one or at leastone and the singular also includes the plural unless it is apparent thatit is meant otherwise.

Upon reading this disclosure, those of ordinary skill in the art willappreciate still additional alternative structural and functionaldesigns for a system and a process for generating composite documentsthrough the disclosed principles herein. Thus, while particularembodiments and applications have been illustrated and described, it isto be understood that the disclosed embodiments are not limited to theprecise construction and components disclosed herein. Variousmodifications, changes and variations, which will be apparent to thoseskilled in the art, may be made in the arrangement, operation anddetails of the method and apparatus disclosed in this document withoutdeparting from the spirit and scope defined in the appended claims.

What is claimed is:
 1. A tangible non-transitory computer-readablemedium storing instructions thereon to implement a composite documentsystem, wherein the instructions, when executed on a processor, causethe processor to: receive model data that describes a three-dimensionalmodel of an object, wherein the three-dimensional model is generatedusing a modeling software and the model data conforms to a non-imageformat of the modeling software; generate a model entity, wherein themodel entity includes the received model data and a reference to asource of the model data; receive web page data that includes contentand a description of the content in a mark-up language; generate a webreference entity, wherein the web reference entity includes the receivedweb page data, a reference to a source of the web page data, and asnapshot of the web page data; generate a composite document thatincludes a first container and a second container; interpret the modeldata in the composite document to generate a visual representation ofthe three-dimensional model in the first container; interpret the webpage data in the composite document to generate a visual representationof the web page data in the second container; retrieve web page datafrom the source of the web page data; compare the retrieved web pagedata to the web page data in the web reference entity; update the secondcontainer in the composite document to include a new version of the webpage data; detect that the three-dimensional model has been updated; andin response to the three-dimensional model having been updated,automatically update the model data in the first container of thecomposite document in accordance with the updated three-dimensionalmodel.
 2. The computer-readable medium of claim 1, wherein the compositedocument includes a plurality of presentation slides, and wherein theinstructions further cause the processor to display the plurality ofpresentation slides via a user interface.
 3. The computer-readablemedium of claim 1, wherein the instructions further cause the processorto provide a set of drawings controls via a user interface, wherein auser utilizes the set of drawings controls to draw two-dimensionalshapes in the composite document.
 4. The computer-readable medium ofclaim 1, wherein the instructions further cause the processor to:receive text from a user interface; and include the received text in thecomposite document.
 5. The computer-readable medium of claim 1, whereinto generate the visual representation of the three-dimensional model,the instructions cause the processor to: interpret the model data todisplay on a user interface the three-dimensional model in a rotatablemode; and receive from the user interface a selection of a view of thethree-dimensional model, wherein the generated visual representation ofthe three-dimensional model corresponds to the selected view.
 6. Thecomputer-readable medium of claim 1, wherein to receive the web pagedata, the instructions cause the processor to: provide an interactiveuser interface to receive from a user a resource identifier, wherein theresource identifier identifies a source web page from which the web pagedata was received; and include the resource identifier in the compositedocument.
 7. The computer-readable medium of claim 6, wherein theinstructions further cause the processor to: receive an indication thatthe source web page has been updated; in response to receiving theindication, automatically retrieve a new version of the web page datafrom the source web page; and update the composite document to includethe new version of the web page data.
 8. The computer-readable medium ofclaim 6, wherein the instructions further cause the processor to displaythe static snapshot of the source web page.
 9. The computer-readablemedium of claim 1, wherein the instructions further cause the processorto: receive document geometry data from a user interface, wherein thedocument geometry includes an indication of a position of the visualrepresentation of the web page data in the composite document; andposition the web page data in the composite document in accordance withthe received document geometry data.
 10. The computer-readable medium ofclaim 9, wherein the instructions further cause the processor to:provide a set of geometry controls via a user interface, wherein a userinteracts with the web page data using the geometry controls to performat least one of: (i) rotate the visual representation of the web pagedata, (ii) scale the visual representation of the web page data, and(iii) crop the visual representation of the web page data.
 11. Thecomputer-readable medium of claim 1, wherein the instructions furthercause the processor to: provide browser controls via a user interface,wherein a user interacts with the web page data using the browsercontrols.
 12. The computer-readable medium of claim 11, wherein: the webpage data is received from a source web page; and the browser controlsinclude at least one of (i) a control to enable a transition forwardrelative to the source web page, (ii) a control to enable a transitionbackward relative to the source web page, and (iii) a control to refreshthe web page data using the source web page.
 13. The computer-readablemedium of claim 1, wherein to generate the visual representation of theweb page data, the instructions cause the processor to render thecontent in the web page data using vector graphics.
 14. Thecomputer-readable medium of claim 1, wherein the static snapshot of theweb page data includes an image of the web page data in a digital imageformat.
 15. A tangible non-transitory computer-readable medium storinginstructions thereon that, when executed on a processor, cause theprocessor to: provide a set of modeling controls via a user interface,wherein a user utilizes the set of modeling controls to draw athree-dimensional model of an object; generate model data descriptive ofthe three-dimensional model in accordance with commands received fromthe user interface; provide a web reference control via the userinterface, wherein the user specifies a source web page using the webreference control; receive web page data, wherein the web page data isretrieved from the source web page and includes content and adescription of the content; generate a web reference entity, wherein theweb reference entity includes the received web page data, a reference toa source of the web page data, and a snapshot of the web page data;generate a composite document using the model data and the web pagedata; retrieve web page data from the source web page; compare theretrieved web page data to the web page data in the web referenceentity; and update the composite document to include the new version ofthe web page data.
 16. The computer-readable medium of claim 15, whereinthe instructions further cause the processor to interpret the web pagedata to generate a visual representation of the web page data.
 17. Thecomputer-readable medium of claim 16, wherein to generate the visualrepresentation of the web page data, the instructions further cause theprocessor to render the content in accordance with the description ofthe content, wherein the description of the content includesinstructions in a mark-up language.
 18. The computer-readable medium ofclaim 16, wherein the instructions further cause the processor toposition the visual representation of the web page data within thecomposite document in accordance with a user command received from theuser interface.
 19. The computer-readable medium of claim 15, whereinthe instructions further cause the processor to store the compositedocument as composite document data on a computer-readable medium,including: store the model data as a first portion of the compositedocument data, and store the web page data as a second portion of thecomposite document data.
 20. The computer-readable medium of claim 15,wherein the instructions further cause the processor to store anidentifier of the source web page as a third portion of the compositedocument data.
 21. The computer-readable medium of claim 15, wherein theinstructions further cause the processor to: receive an update to thethree-dimensional model from the user interface; and automaticallyupdate the composite document using the update to the three-dimensionalmodel.
 22. The computer-readable medium of claim 15, wherein theinstructions further cause the processor to: receive an indication thatthe web page data has changed at the source web page data; in responseto receiving the indication, automatically retrieve a new version of theweb page data from the source web page; and update the compositedocument using the new version of the web page data.
 23. A method in acomputing device, the method comprising: receiving model data thatconforms to a non-image format, wherein the model data is stored on acomputer-readable medium and the model data describes athree-dimensional model; receiving text interpretable as a universalresource locator (URL); receiving web page data from a source web page,wherein the web page data includes content and a description of thecontent and wherein the text interpretable as a URL identifies thesource web page; generating a web reference entity, wherein the webreference entity includes the received web page data, a snapshot of theweb page data, and the text interpretable as a URL; generating acomposite document using the model data and the web page data;retrieving web page data from the source web page; comparing theretrieved web page data to web page data in the web reference entity;updating the composite document to include a new version of the web pagedata and causing the composite document to be displayed on a userinterface, including: generating a visual representation of thethree-dimensional model using the model data, and generating a visualrepresentation of the source web page using the web page data.
 24. Themethod of claim 23, wherein generating the composite document includesrendering the content in the web page data in accordance with theinstructions in the web page data.
 25. The method of claim 23, furthercomprising causing the composite document to be stored in acomputer-readable memory, including storing the model data and the webpage data in the composite document.
 26. The method of claim 25, furthercomprising storing an identifier of the source web page in the compositedocument.
 27. The method of claim 23, further comprising: receivingpositioning data corresponding to the web page data; and positioning theweb page data within the composite document in accordance with thereceived positioning data.